Semiconductor integrated circuit technology has rapidly progressed. For example, in the case of dynamic memory, memory chips with a capacity of 1 though 4 Mbit have been already put into mass production and ultra large integration memory chips such as 16 Mb or 64 Mb memory are now being developed. Such an ultra large integration technology is applied to a logic circuit as well as a memory circuit, and a variety of functional logic integrated circuits represented by 32 bit CPU or 64 bit CPU are also under development.
In these logic circuits, an arithmetic operation is carried out by using digital signals, i.e., binary signals composed of 1 and 0. For example, a Neumann method is adopted for a computer, where commands are executed one by one according to a predetermined program. Although simple numerical calculations can be carried out at very high speed by this method, it takes a lot of time to perform the pattern recognition or image processing. Furthermore, this method is not suitable for information processing such as association of ideas, memorizing and learning, which is mankind's strongest point. In spite of a lot of research and development activities for software technology, notable results have not been produced yet.
There has been another stream of research to get rid of these difficulties at once and then to construct a computer which imitates arithmetic operations of the brain, i.e. neuron circuit computer (neural computer) by studying brain functions of living things. This kind of research began in the 1940s and has become very active in last several years based on the fact that the progress in LSI technology may make it possible to realize the hardware of neuron computer.
However, the present semiconductor LSI technology still has too many problems to put it into practice. This is described more concretely. For example, in order to make the hardware with the function of one human neuron cell (neuron), a circuit must be constructed by combining a lot of semiconductor elements. In other words, it is very difficult to construct the practical number of neurons on a semiconductor chip.
Before a detailed explanation of the invention is given, the function which is required for one neuron and the difficulties associated with the construction of a neuron using current technologies are described.
FIG. 19(a) is a schematic representation illustrating the function of a neuron cell, i.e., neuron, which was proposed by McCumllock and Pitts as a mathematical model of neuron (Bull. Math. Biophys., Vol.5, p.115 (1943)). At present, the studies are being carried out actively to construct a neuron computer by realizing this model with semiconductor circuits. V.sub.1, V.sub.2, V.sub.3, . . . , V.sub.n are n input signals defined as, for example, magnitudes of voltages, and correspond to signals transferred from other neurons. W.sub.1, W.sub.2, W.sub.3, . . . , W.sub.n are coefficients representing the coupling strengths between neurons, and are biologically called synapse couplings. The function of this neuron is simple. When the value Z, linear sum of the product of each input V.sub.i and weight W.sub.i (i=1-n), becomes larger than a predetermined threshold value V.sub.th*, the neuron outputs 1; on the other hand, 0 when Z is less than V.sub.th *. The numerical expression is as follows: ##EQU1##
FIG. 19(b) shows the relationship between Z and V.sub.out. The output is 1 when Z is large enough as compared with V.sub.th* and 0 when Z is small enough.
Next, an example of the circuits to realize the above-mentioned function using conventional semiconductor technology 1a shown in FIG. 20. In the figure, 102-1, 102-2 and 102-3 denote operational amplifiers. FIG. 20(a) shows a circuit to obtain Z by adding the product of input signal V.sub.i (i=1-n) and weight Wi. I.sub.i denotes electric current flowing through R.sub.i. From I.sub.i -V.sub.i /R.sub.i, ##EQU2## the output voltage V.sub.a of the operational amplifier 102-1 is given by ##EQU3## Since I.sub.b is given by -V.sub.s /R.sub.1, I.sub.a and I.sub.b have the same magnitude (I.sub.a =I.sub.b) and the opposite direction of flow, leading to the expression: ##EQU4## By the comparison between Eqs. (1') and (4'), the weight coefficient W.sub.1 is found to be R/R.sub.i and therefore determined by the resistance. The circuit shown in FIG. 20(a) is a circuit to generate the voltage representing the linear sum of input signals obtained by summing up electric currents. FIG. 20(b) is an example of circuit to convert the value of Z into V.sub.out, where Z is connected to a non-inversion input terminal of operational amplifier 102-3. Since an operational amplifier is an amplifier having a large voltage amplification (gain), V.sub.out =V.sup.+ when Z&gt;E.sub.0 and V.sub.out =V.sup.- when Z&lt;E.sub.0, as shown in FIG. 20(c). Here, V.sup.+ and V.sup.- are the maximum and the minimum values of outputs which are determined by power supply voltage supplied to the operational amplifier. The value of V.sub.TH* can be changed by varying the voltage E.sub.0 applied to non-inversion terminal. One of the problems of the circuit shown in FIGS. 20(a) and (b) is that a lot of semiconductor elements are required to construct a neuron. Three operational amplifiers are used in the circuit of the figure and therefore 30 transistors are necessary since at least 10 transistors are usually required to construct one operational amplifier. And since the sum operation is made on the basis of electric current mode, a large amount of current always flows, resulting in large power dissipation. Namely, one neuron not only occupies a large area on a chip but also dissipates much power. Therefore, it is difficult to attain large scale integration. Even if large scale integration can be attained by shrinking the dimensions of transistor, it is almost impossible to construct a practical integrated circuit because of high density of the power dissipation.
The unit of brain is simplified and then further described in FIG. 29. In the figure, 2921a, 2921b and 2921c are neurons. 2922a, 2922b and 2922c are nerve fibers. 2923a, 2923b and 2923c are called as synapse couplings. For example, a signal transmitted through nerve fiber 2922a is multiplied by weight W.sub.a, at synapse coupling 2923a and the weighted signal is inputted to neuron 2921a. Neuron 2921a sums up inputted signals and is activated to output signal (namely, the neuron gets fired) when the sum is larger than a threshold value. On the other hand, the neuron outputs no signal when the sum is less than the threshold value.
Such phenomena as arithmetic operation, transmission of signal, and weight multiplication are carried out electrochemically in a real brain, and signals are transferred and processed as electric signals. The learning process of humans is correlated to the change in weights of synapse couplings. The weight is gradually modified to the optimum value to have correct output for various combinations of inputted signals. In other words, the knowledge is stored in the brain as weights of synapse couplings.
A number of neurons are mutually connected, forming a layer and it is known that six layers are stacked in the human brain. In realizing a neuron computer, it is the most important matter to construct LSI system with above-mentioned structure and function.
FIGS. 30(a) and (b) are schematic representations illustrating the function of a nerve cell, i.e., neuron, which were proposed by McCumllock and Pitts as a mathematical model of neuron (Bull. Math. Biophys., Vol.5, p.115 (1943)). At present, the studies are being carried out actively to construct a neuron computer by realizing this model with semiconductor circuits. V.sub.I, V.sub.2, V.sub.3, . . . , V.sub.n are n input signals defined as, for example, magnitudes of voltages, and correspond to signals, transferred from other neurons. W.sub.1, W.sub.2, W.sub.3, . . . , W.sub.n are coefficients representing the coupling strengths between neurons and are biologically called synapse couplings. The function of this neuron is that when the value Z, the linear sum of products of input V.sub.i and respective weight W.sub.i, (i=1 through n), becomes larger than a predetermined threshold value V.sub.TH*, the neuron outputs 1; on the other hand, 0 when Z is less than V.sub.TH*. The numerical expression is given as follows: ##EQU5##
FIG. 30(b) shows the relationship Between Z and V.sub.out. The output is 1 when Z is large enough as compared with V.sub.TH*, and 0 when Z is small enough.
In order to construct the neuron using conventional transistors, a large number of transistors is required. Furthermore, since the sum operation is carried out after signals are converted to electric current, a large amount of current flows, resulting in large power dissipation. This means the integration of neurons is substantially impossible.